E-insensitive Unsupervised Learning

One of the major paradigms for unsupervised learning in Artificial Neural Networks is Hebbian learning. The standard implementations of Hebbian learning are optimal under the assumptions of Gaussian noise in a data set. We derive e-insensitive Hebbian learning based on minimising the least absolute error in a compressed data set and show that the learning rule is equivalent to the Principal Component Analysis (PCA) networks' learning rules under a variety of conditions

A novel construction of connectivity graphs for clustering and visualization

We [5, 6] have recently investigated several families of clustering algorithms. In this paper, we show how a novel similarity function can be integrated into one of our algorithms as a method of performing clustering and show that the resulting method is superior to existing methods in that it can be shown to reliably find a globally optimal clustering rather than local optima which other methods often find. We discuss some of the current difficulties with using connectivity graphs for solving clustering problems, and then we introduce a new algorithm to build the connectivity graphs. We compare this new algorithm with some famous algorithms used to build connectivity graphs. The new algorithm is shown to be superior to those in the current literature. We also extend the method to perform topology preserving mappings and show the results of such mappings on artificial and real data

Online clustering algorithms

We introduce a set of clustering algorithms whose performance function is such that the algorithms overcome one of the weaknesses of K-means, its sensitivity to initial conditions which leads it to converge to a local optimum rather than the global optimum. We derive online learning algorithms and illustrate their convergence to optimal solutions which K-means fails to find. We then extend the algorithm by underpinning it with a latent space which enables a topology preserving mapping to be found. We show visualisation results on some standard data sets

Clustering with alternative similarity functions

We [6, 7] have recently investigated several families of clustering algorithms. In this paper, we show how a novel similarity function can be integrated into one of our algorithms as a method of performing clustering and show that the resulting method is superior to existing methods in that it canbe shown to reliably find a globally optimal clustering rather than local optima which other methods often find. We also extend the method to perform topology preserving mappings and show the results of such mappings on artificial and real data

Reservoir computing and data visualisation

We consider the problem of visualisation of high dimensional multivariate time series. A data analyst in creating a two dimensional projection of such a time series might hope to gain some intuition into the structure of the original high dimensional data set. We review a method for visualising time series data using an extension of Echo State Networks (ESNs). The method uses the multidimensional scaling criterion in order to create a visualisation of the time series after its representation in the reservoir of the ESN. We illustrate the method with two dimensional maps of a financial time series. The method is then compared with a mapping which uses a fixed latent space and a novel objective function

Nonlinear Boosting Projections for Ensemble Construction

In this paper we propose a novel approach for ensemble construction based on the use of nonlinear projections to achieve both accuracy and diversity of individual classifiers. The proposed approach combines the philosophy of boosting, putting more effort on difficult instances, with the basis of the random subspace method. Our main contribution is that instead of using a random subspace, we construct a projection taking into account the instances which have posed most difficulties to previous classifiers. In this way, consecutive nonlinear projections are created by a neural network trained using only incorrectly classified instances. The feature subspace induced by the hidden layer of this network is used as the input space to a new classifier. The method is compared with bagging and boosting techniques, showing an improved performance on a large set of 44 problems from the UCI Machine Learning Repository. An additional study showed that the proposed approach is less sensitive to noise in the data than boosting method

Topological mappings of video and audio data

We review a new form of self-organizing map which is based on a nonlinear projection of latent points into data space, identical to that performed in the Generative Topographic Mapping (GTM).1 But whereas the GTM is an extension of a mixture of experts, this model is an extension of a product of experts.2 We show visualisation and clustering results on a data set composed of video data of lips uttering 5 Korean vowels. Finally we note that we may dispense with the probabilistic underpinnings of the product of experts and derive the same algorithm as a minimisation of mean squared error between the prototypes and the data. This leads us to suggest a new algorithm which incorporates local and global information in the clustering. Both ot the new algorithms achieve better results than the standard Self-Organizing Map

Connectionist Techniques for the identification and suppression of interfering underlying factors

We consider the difficult problem of identification of independent causes from a mixture of them when these causes interfere with one another in a particular manner: those considered are visual inputs to a neural network system which are created by independent underlying causes which may occlude each other. The prototypical problem in this area is a mixture of horizontal and vertical bars in which each horizontal bar interferes with the representation of each vertical bar and vice versa. Previous researchers have developed artificial neural networks which can identify the individual causes; we seek to go further in that we create artificial neural networks which identify all the horizontal bars from only such a mixture. This task is a necessary precursor to the development of the concept of "horizontal" or "vertical"

A comparison of Kernel methods for instantiating case based reasoning systems

Instance based reasoning systems and in general case based reasoning systems are normally used in problems for which it is difficult to define rules. Instance based reasoning is the term which tends to be applied to systems where there are a great amount of data (often of a numerical nature). The volume of data in such systems leads to difficulties with respect to case retrieval and matching. This paper presents a comparative study of a group of methods based on Kernels, which attempt to identify only the most significant cases with which to instantiate a case base. Kernels were originally derived in the context of Support Vector Machines which identify the smallest number of data points necessary to solve a particular problem (e.g. regression or classification). We use unsupervised Kernel methods to identify the optimal cases to instantiate a case base. The efficiencies of the Kernel models measured as Mean Absolute Percentage Error are compared on an oceanographic problem

Unsupervised neural method for temperature forecasting

This article presents the results of using a novel Negative Feedback Artificial Neural Network for extraction of models of the thermal structure of oceanographic water masses and to forecast time series in real time. The results obtained using this model are compared with those obtained using a Linear Regression and an ARIMA model. The article presents the Negative Feedback Artificial Neural Network, shows how it extracts the model behind the data set and discuses the Artificial Neural Network's forecasting abilities